POMDP Compression and Decomposition via Belief State Analysis

Partially observable Markov decision process (POMDP) is a commonly adopted mathematical framework for solving planning problems in stochastic environments. However, computing the optimal policy of POMDP for large-scale problems is known to be intractable, where the high dimensionality of the underlying belief state space is one of the major causes. Our research focuses on studying two different paradigms, namely POMDP compression and POMDP decomposition, for addressing the POMDP’s tractability issue. We proposed a novel orthogonal NMF hybrid approach to compress the POMDP problem. For POMDP decomposition, we propose a clustering criteria function which takes into the account the temporal difference of belief sample points for partitioning the belief space. We evaluated the proposed approaches based on a set of benchmark problems. We demonstrated that belief compression and belief clustering are both useful in effectively reducing the cost for computing policies, and yet with the quality of the policies more or less retained.